Towards predicting web searcher gaze position from mouse movements.

Using the past to score the present extending term weighting models through revision history analysis.

Deconstructing Interaction Dynamics in Knowledge Sharing Communities.

Ready to buy or just browsing%3F detecting web searcher goals from interaction data.
An improved understanding of the relationship between search intent, result quality, and searcher behavior is crucial for improving the effectiveness of web search. While recent progress in user behavior mining has been largely focused on aggregate server-side click logs, we present a new class of search behavior models that also exploit fine-grained user interactions with the search results. We show that mining these interactions, such as mouse movements and scrolling, can enable more effective detection of the user’s search goals. Potential applications include automatic search evaluation, improving search ranking, result presentation, and search advertising. We describe extensive experimental evaluation over both controlled user studies, and logs of interaction data collected from hundreds of real users. The results show that our method is more effective than the current state-of-the-art techniques, both for detection of searcher goals, and for an important practical application of predicting ad clicks for a given search session.
Exploring searcher interactions for distinguishing types of commercial intent.

Unsupervised query segmentation using click data preliminary results.

Classifying and Characterizing Query Intent.
Abstract. Understanding the intent underlying user queries may help personalize search results and improve user satisfaction. In this paper, we develop a methodology for using ad clickthrough logs, query specific information, and the content of search engine result pages to study characteristics of query intents, specially commercial intent. The findings of our study suggest that ad clickthrough features, query features, and the content of search engine result pages are together effective in detecting query intent. We also study the effect of query type and the number of displayed ads on the average clickthrough rate. As a practical application of our work, we show that modeling query intent can improve the accuracy of predicting ad clickthrough for previously unseen queries. 1
Beyond session segmentation predicting changes in search intent with client%2Dside user interactions.

EMU the emory user behavior data management system for automatic library search evaluation.

Beyond session segmentation predicting changes in search intent with client%2Dside user interactions.



Learning to recognize reliable users and content in social media with coupled mutual reinforcement.
Community Question Answering (CQA) has emerged as a popular forum for users to pose questions for other users to answer. Over the last few years, CQA portals such as Naver and Yahoo! Answers have exploded in popularity, and now provide a viable alternative to general purpose Web search. At the same time, the answers to past questions submitted in CQA sites comprise a valuable knowledge repository which could be a gold mine for information retrieval and automatic question answering. Unfortunately, the quality of the submitted questions and answers varies widely- increasingly so that a large fraction of the content is not usable for answering queries. Previous approaches for retrieving relevant and high quality content have been proposed, but they require large amounts of manually labeled data – which
Estimating Ad Clickthrough Rate through Query Intent Analysis.
Clickthrough rate and cost-per-click are known to be among the factors that impact the rank of an ad shown on a search result page. Hence, search engines can benefit from estimating ad clickthrough in order to determine the quality of ads and maximize their revenue. In this paper, a methodology is developed to estimate ad clickthrough rate by exploring user queries and clickthrough logs. As we demonstrate, the average ad clickthrough rate depends to a substantial extent on the rank position of ads and on the total number of ads displayed on the page. This observation is utilized by a baseline model to calculate the expected clickthrough rate for various ads. We further study the impact of query intent on the clickthrough rate, where query intent is predicted using a combination of query features and the content of search engine result pages. The baseline model and the query intent model are compared for the purpose of calculating the expected ad clickthrough rate. Our findings suggest that factors such as the rank of an ad, the number of ads displayed on the result page, and query intent are effective in estimating ad clickthrough rates. 1.
In the Mood to Click%3F Towards Inferring Receptiveness to Search Advertising.
Abstract—We present a method for modeling, and automatically inferring, the current interest of a user in search advertising. Our task is complementary to that of predicting ad relevance or commercial intent of a query in the aggregate, since the user intent may vary significantly for the same query. To achieve this goal, we develop a fine-grained user interaction model for inferring searcher receptiveness to advertising. We show that modeling the search context and behavior can significantly improve the accuracy of ad clickthrough prediction for the current user, compared to the existing state-of-the-art classification methods that do not model this additional sessionlevel contextual and interaction information. In particular, our experiments over thousands of search sessions from hundreds of real users demonstrate that our model is more effective at predicting ad clickthrough within the same search session. Our work has other potential applications, such as improving search interface design (e.g., varying the number or type of ads) based on user interest, and behavioral targeting (e.g., identifying users interested in immediate purchase). I.
The search and social media workshop at SIGIR 2009.

Modeling information%2Dseeker satisfaction in community question answering.

Proceeding of the 2008 ACM Workshop on Search in Social Media

You%27ve Got Answers Towards Personalized Models for Predicting Success in Community Question Answering.
Question answering communities such as Yahoo! Answers have emerged as a popular alternative to general-purpose web search. By directly interacting with other participants, information seekers can obtain specific answers to their questions. However, user success in obtaining satisfactory answers varies greatly. We hypothesize that satisfaction with the contributed answers is largely determined by the asker’s prior experience, expectations, and personal preferences. Hence, we begin to develop personalized models of asker satisfaction to predict whether a particular question author will be satisfied with the answers contributed by the community participants. We formalize this problem, and explore a variety of content, structure, and interaction features for this task using standard machine learning techniques. Our experimental evaluation over thousands of real questions indicates that indeed it is beneficial to personalize satisfaction predictions when sufficient prior user history exists, significantly improving accuracy over a “one-size-fits-all ” prediction model. 1
A few bad votes too many%3F towards robust ranking in social media.



Predicting information seeker satisfaction in community question answering.

Exploring mouse movements for inferring query intent.

Exploring question subjectivity prediction in community QA.

On the evolution of the yahoo%21 answers QA community.

Finding high%2Dquality content in social media.
ABSTRACT: The quality of user-generated content varies drastically from excellent to abuse and spam. As the availability of such content increases, the task of identifying high-quality content in sites based on user contributions—social media sites—becomes increasingly important. Social media in general exhibit a rich variety of information sources: in addition to the content itself, there is a wide array of non-content information available, such as links between items and explicit quality ratings from members of the community. In this paper we investigate methods for exploiting such community feedback to automatically identify high quality content. As a test case, we focus on Yahoo! Answers, a large community question answering portal that is particularly rich in the amount and types of content and social interactions available in it. We introduce a general classification framework for combining the evidence from different sources of information that can be tuned automatically for a given social media type and quality definition. In particular, for the community question answering domain, we show that our system is able to separate high-quality items from the rest with an accuracy close to that of humans. 1
Finding the right facts in the crowd factoid question answering over social media.

Discovering semantic biomedical relations utilizing the Web.

Discovering authorities in question answer communities by using link analysis.
Question-Answer portals such as Naver and Yahoo! Answers are quickly becoming rich sources of knowledge on many topics which are not well served by general web search engines. Unfortunately, the quality of the submitted answers is uneven, ranging from excellent detailed answers to snappy and insulting remarks or even advertisements for commercial content. Furthermore, user feedback for many topics is sparse, and can be insufficient to reliably identify good answers from the bad ones. Hence, estimating the authority of users is a crucial task for this emerging domain, with potential applications to answer ranking, spam detection, and incentive mechanism design. We present an analysis of the link structure of a general-purpose question answering community to discover authoritative users, and promising experimental results over a dataset of more than 3 million answers from a popular community QA site. We also describe structural differences between question topics that correlate with the success of link analysis for authority discovery. Categories and Subject Descriptors:
Domain Ontology Construction from Biomedical Text.
(UMLS) is a very large ontology of biomedical and health data. In order to be used effectively for knowledge processing, it needs to be customized to a specific domain. In this paper, we present techniques to automatically discover domain-specific concepts, discover relationships between these concepts, build a context map from these relationships, link these domain concepts with the bestmatching concept identifiers in UMLS using our context map and UMLS concept trees, and finally assign categories to the discovered relationships. This specific domain ontology of terms and relationships using evidential information can serve as a basis for applications in analysis, reasoning and discovery of new relationships. We have automatically built an ontology for the Nuclear Cardiology domain as a testbed for our techniques.
Hits on question answer portals exploration of link analysis for author ranking.%
Question-Answer portals such as Naver and Yahoo! Answers are growing in popularity. However, despite the increased popularity, the quality of answers is uneven, and while some users usually provide good answers, many others often provide bad answers. Hence, estimating the authority, or the expected quality of users, is a crucial task for this emerging domain, with potential applications to answer ranking and to incentive mechanism design. We adapt a powerful link analysis methodology from the web domain as a first step towards estimating authority in Question Answer portals. Our experimental results over more than 3 million answers from Yahoo! Answers are promising, and warrant further exploration along the lines outlined in this poster.
The influence of caption features on clickthrough patterns in web search.

Hits on question answer portals exploration of link analysis for author ranking.%
Question-Answer portals such as Naver and Yahoo! Answers are growing in popularity. However, despite the increased popularity, the quality of answers is uneven, and while some users usually provide good answers, many others often provide bad answers. Hence, estimating the authority, or the expected quality of users, is a crucial task for this emerging domain, with potential applications to answer ranking and to incentive mechanism design. We adapt a powerful link analysis methodology from the web domain as a first step towards estimating authority in Question Answer portals. Our experimental results over more than 3 million answers from Yahoo! Answers are promising, and warrant further exploration along the lines outlined in this poster.
Question Answering over Implicitly Structured Web Content.
Implicitly structured content on the Web such as HTML tables and lists can be extremely valuable for web search, question answering, and information retrieval, as the implicit structure in a page often reflects the underlying semantics of the data. Unfortunately, exploiting this information presents significant challenges due to the immense amount of implicitly structured content on the web, lack of schema information, and unknown source quality. We present TQA, a web-scale system for automatic question answering that is often able to find answers to real natural language questions from the implicitly structured content on the web. Our experiments over more than 200 million structures extracted from a partial web crawl demonstrate the promise of our approach. 1
Towards a query optimizer for text%2Dcentric tasks.
Text is ubiquitous and, not surprisingly, many important applications rely on textual data for a variety of tasks. As a notable example, information extraction applications derive structured relations from unstructured text; as another example, focused crawlers explore the Web to locate pages about specific topics. Execution plans for text-centric tasks follow two general paradigms for processing a text database: either we can scan, or “crawl, ” the text database or, alternatively, we can exploit search engine indexes and retrieve the documents of interest via carefully crafted queries constructed in task-specific ways. The choice between crawl- and query-based execution plans can have a substantial impact on both execution time and output “completeness ” (e.g., in terms of recall). Nevertheless, this choice is typically ad hoc and based on heuristics or plain intuition. In this article, we present fundamental building blocks to make the choice of execution plans for text-centric tasks in an informed, cost-based way. Towards this goal, we show how to analyze query- and crawl-based plans in terms of both execution time and output completeness. We adapt results from random-graph theory and statistics to develop a rigorous cost model for the execution plans. Our cost model reflects the fact that the performance of the plans depends on fundamental task-specific properties of the underlying text databases. We identify these properties and present
Identifying %22best bet%22 web search results by mining past user behavior.
In the first part of the paper we consider the problem of dynamically apportioning resources among a set of options in a worst-case on-line framework. The model we study can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting. We show that the multiplicative weightupdate rule of Littlestone and Warmuth [20] can be adapted to this model yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. We show how the resulting learning algorithm can be applied to a variety of problems, including gambling, multiple-outcome prediction, repeated games and prediction of points in R  n  . In the second part of the paper we apply the multiplicative weight-update technique to derive a new boosting algorithm. This boosting algorithm does not require any prior knowledge about the performance of the weak learning algorithm. We also study generalizations of...
Confidence Estimation Methods for Partially Supervised Information Extraction.

Improving web search ranking by incorporating user behavior information.

Learning user interaction models for predicting web search result preferences.
Evaluating user preferences of web search results is crucial for search engine development, deployment, and maintenance. We present a real-world study of modeling the behavior of web search users to predict web search result preferences. Accurate modeling and interpretation of user behavior has important applications to ranking, click spam detection, web search personalization, and other tasks. Our key insight to improving robustness of interpreting implicit feedback is to model query-dependent deviations from the expected “noisy” user behavior. We show that our model of clickthrough interpretation improves prediction accuracy over state-of-the-art clickthrough methods. We generalize our approach to model user behavior beyond clickthrough, which results in higher preference prediction accuracy than models based on clickthrough information alone. We report results of a large-scale experimental evaluation that show substantial improvements over published implicit feedback interpretation methods.
To search or to crawl%3F towards a query optimizer for text%2Dcentric tasks.

Web Information Extraction and User Modeling Towards Closing the Gap.
Web search engines have become the primary method of accessing information on the web. Billions of queries are submitted to major web search engines, reflecting a wide range of information needs. While significant progress has been made on improving the relevance of the results, web search process often remains a frustrating experience. At the same time, web information extraction has seen tremendous progress, such that knowledge bases of millions of facts extracted from the web are now a reality. Yet it is not clear how effectively these knowledge bases support common user information needs. We posit that a key for web information extraction to significantly impact the web search experience is to connect the extraction process with user modeling, particularly with automatic methods for inferring user information needs and anticipated interaction patterns. In this paper we overview some recent efforts for user modeling and inferring user preferences in the context of closing the gap between web information extraction and user modeling. 1
Predicting accuracy of extracting information from unstructured text collections.
Exploiting lexical and semantic relationships in large unstructured text collections can significantly enhance managing, integrating, and querying information locked in unstructured text. Most notably, named entities and relations between entities are crucial for effective question answering and other information retrieval and knowledge management tasks. Unfortunately, the success in extracting these relationships can vary for different domains, languages, and document collections. Predicting extraction performance is an important step towards scalable and intelligent knowledge management, information retrieval and information integration. We present a general language modeling method for quantifying the difficulty of information extraction tasks. We demonstrate the viability of our approach by predicting performance of real world information extraction tasks, Named Entity recognition and Relation Extraction.

Analysis of factoid questions for effective relation extraction.


Scaling Information Extraction to Large Document Collections.
Information extraction and text mining applications are just beginning to tap the immense amounts of valuable textual information available online. In order to extract information from millions, and in some cases, billions of documents, different solutions to scalability emerged. We review key approaches for scaling up information extraction, including using general-purpose search engines as well as indexing techniques specialized for information extraction applications. Scalable information extraction is an active area of research, and we highlight some of the opportunities and challenges in this area that are relevant to the database community. 1
Mining reference tables for automatic text segmentation.
Automatically segmenting unstructured text strings into structured records is necessary for importing the information contained in legacy  sources and text collections into a data warehouse for subsequent querying, analysis, mining and integration. In this paper, we mine tables present in data warehouses and relational databases to develop an automatic segmentation system. Thus, we overcome limitations of existing supervised text segmentation approaches, which require comprehensive manually labeled training data. Our segmentation system is robust, accurate, and efficient, and requires no additional manual effort. Thorough evaluation on real datasets demonstrates the robustness and accuracy of our system, with segmentation accuracy exceeding state of the art supervised approaches.
Combining Text Mining and Sequence Analysis to Discover Protein Functional Regions.

Learning to find answers to questions on the Web.

Extracting Relations from XML Documents.
XML is becoming a prevalent format for data exchange. Many XML documents have complex schemas that are not always known, and can vary widely between information sources and applications. In contrast, database applications rely mainly on the flat relational model. We propose a novel, partially supervised approach for extracting userdefined relations from XML documents with unknown schema. The extracted relations can be directly used by an RDBMS, or utilized for information integration or data mining tasks. Our method attempts to automatically capture the lexical and structural features that indicate the relevant portions of the input document, based on a few user-annotated examples. This information can then be used to extract the relation of interest from documents with schemas potentially di#erent from the training examples. We present preliminary experiments showing that our method could be capable of extracting the target relation from XML documents even in the presence of significant variations in the document schemas.
Querying Text Databases for Efficient Information Extraction.
A wealth of information is hidden within unstructured text. This information is often best exploited in structured or relational form, which is suited for sophisticated query processing, for integration with relational databases, and for data mining. Current information extraction techniques extract relations from a text database by examining every document in the database, or use filters to select promising documents for extraction. The exhaustive scanning approach is not practical or even feasible for large databases, and the current filtering techniques require human involvement to maintain and to adopt to new databases and domains. In this paper, we develop an automatic query-based technique to retrieve documents useful for the extraction of user-defined relations from large text databases, which can be adapted to new domains, databases, or target relations with minimal human effort. We report a thorough experimental evaluation over a large newspaper archive that shows that we significantly improve the efficiency of the extraction process by focusing only on promising documents.
Querying Text Databases for Efficient Information Extraction.
A wealth of information is hidden within unstructured text. This information is often best exploited in structured or relational form, which is suited for sophisticated query processing, for integration with relational databases, and for data mining. Current information extraction techniques extract relations from a text database by examining every document in the database, or use filters to select promising documents for extraction. The exhaustive scanning approach is not practical or even feasible for large databases, and the current filtering techniques require human involvement to maintain and to adopt to new databases and domains. In this paper, we develop an automatic query-based technique to retrieve documents useful for the extraction of user-defined relations from large text databases, which can be adapted to new domains, databases, or target relations with minimal human effort. We report a thorough experimental evaluation over a large newspaper archive that shows that we significantly improve the efficiency of the extraction process by focusing only on promising documents.
Extracting synonymous gene and protein terms from biological literature.
Motivation: Genes and proteins are often associated with multiple names. More names are added as new functional or structural information is discovered. Because authors can use any one of the known names for a gene or protein, information retrieval and extraction would benefit from identifying the gene and protein terms that are synonyms of the same substance.
QXtract A Building Block for Efficient Information Extraction from Plain%2DText Databases.

Modeling Query%2DBased Access to Text Databases.
require access to such databases often resort to querying to extract relevant documents because of two main reasons. First, some text databases on the web are not &#034;crawlable,&#034; and hence the only way to retrieve their documents is via querying. Second, applications often require only a small fraction of a database&#039;s contents, so retrieving relevant documents via querying is an attractive choice from an efficiency viewpoint, even for crawlable databases. Often an application&#039;s query-based strategy starts with a small number of user-provided queries. Then, new queries are extracted --in an application-dependent way-- from the documents in the initial query results, and the process iterates. The success of this common type of strategy relies on retrieved documents &#034;contributing&#034; new queries. If new documents fail to produce new queries, then the process might stall before all relevant documents are retrieved. In this paper, we develop a graph-based &#034;reachability&#034; metric that allows to characterize when an application&#039;s query-based strategy will successfully &#034;reach&#034; all documents that the application needs. We complement our metric with an efficient sampling-based technique that accurately estimates the reachability associated with a text database and an application&#039;s query-based strategy. We report preliminary experiments backing the usefulness of our metric and the accuracy of the associated estimation technique over real text databases and for two applications.
Snowball A Prototype System for Extracting Relations from Large Text Collections.


Learning search engine specific query transformations for question answering.
We introduce a method for learning query transformations that improves the ability to retrieve answers to questions from an information retrieval system. During the training stage the method involves automatically learning phrase features for classifying questions into different types, automatically generating candidate query transformations from a training set of question/answer pairs, and automatically evaluating the candidate transforms on target information retrieval systems such as real-world general purpose search engines. At run time, questions are transformed into a set of queries, and re-ranking is performed on the documents retrieved. We present a prototype search engine, Tritus, that applies the method to web search engines. Blind evaluation on a set of real queries from a web search engine log shows that the method significantly outperforms the underlying web search engines as well as a commercial search engine specializing in question answering.

Combining Strategies for Extracting Relations from Text Collections.
Abstract Text documents often contain valuable structured datathat is hidden in regular English sentences. This data is best exploited if available as a relational table that wecould use for answering precise queries or for running data mining tasks. Our Snowball system extracts theserelations from document collections starting with only a handful of user-provided example tuples. Based onthese tuples, Snowball generates patterns that are used, in turn, to find more tuples. In this paper we introduce anew pattern and tuple generation scheme for Snowball, with different strengths and weaknesses than those ofour original system. We also show preliminary results on how we can combine the two versions of Snowballto extract tuples more accurately. 1 Introduction Text documents often hide valuable structured data.For example, a collection of newspaper articles might contain information on the location of the headquartersof a number of organizations. The web contains millions of pages whose text hides data that would bebest exploited in structured form.
